Critiquing Human Judgment Using Knowledge-Acquisition Systems

tapered or squared? Should the ears be covered? Should the top be just trimmed or shortened? Most barbers only ask a few questions. Often, they assume one of a few standard models of the customer’s hairstyle and pursue this model until they are given directions to the contrary. Given no direction, they still give a haircut. Even with directions, they must make numerous, perhaps hundreds, of intermediate decisions that drive the haircut toward a model in their minds. Also, far too often, they inject a personal bias or interpretation, and although they are expert hairstylists, this bias results in a surprise—a haircut that the customer did not ask for or expect. Scene 2: A student signs up for a Master’s degree program and asks his adviser (1) to release the prerequisite course requirements because he had similar courses in the undergraduate curriculum, (2) to waive several required courses because of on-the-job experience in these fields, and (3) to permit several electives not normally taken by students in this curriculum. By asking a series of standard It is evident that when the instances on one side of a question are more likely to be remembered and recorded than those on the other, especially if there be any strong motive to preserve the memory of the first, but not of the latter, these last are likely to be overlooked, and escape the observations of the mass of mankind. —John Stuart Mill The fundamental premise being pursued in this article concerns the need for a strong model of a domain-trained knowledge engineer (that is, a knowledge worker) to be embedded in a questionasking system if this system is to be effective in having a dialog with and extracting usable knowledge from the user. The user is defined here as a nonprogramming domain expert. An expert, in turn, is an individual with some know-how in a given domain. Consider the following: Scene 1: A person walks into a barbershop and asks for a haircut. What sort of haircut does the barber give? The style depends, in part, on the sex, age, and desires of the customer. How long should the sideburns be? The fundamental premise . . . concerns the need for a strong model of a domaintrained knowledge engineer . . . to be embedded in a questionasking system . . . Articles